Coherent secondary rays for reflections in hybrid ray tracing

ABSTRACT

Hybrid ray tracing method for generating photorealistic reflections while keeping coherence of secondary rays. The coherence is achieved by decoupling secondary rays from primary rays, handling bundles of secondary rays with collective origin and collective destination, attaining high parallelism and high utilization of GPU.

CROSS-REFERENCE TO RELATED CASES

The present application is a continuation of U.S. application Ser. No.16/874,705, filed on May 15, 2020, entitled “CREATING COHERENT SECONDARYRAYS FOR REFLECTIONS IN HYBRID RAY TRACING”, which is a continuation ofU.S. application Ser. No. 16/844,681 filed on Apr. 9, 2020, entitled:“METHOD FOR CONSTRUCTING and TRAVERSING ACCELERATING STRUCTURES”, whichclaims priority from U.S. Provisional Application No. 62/957,157 filedon Jan. 4, 2020, entitled: “Ray Traced Look-ahead Navigation in VideoGames”, which are all incorporated hereby by reference.

The present application refers to the U.S. application Ser. No.16/662,657, filed Oct. 24, 2019, entitled: “Method for Non-PlanarSpecular Reflections in Hybrid Ray Tracing” (now U.S. Pat. No.10,699,468), and incorporated hereby by reference, which teaches areal-time hybrid ray tracing method for non-planar specular reflections.The high complexity of a non-planar surface is reduced to low complexityof multiple small planar surfaces. Advantage is taken of the planarnature of triangles that comprise building blocks of a non-planarsurface. All secondary rays bouncing from a given surface triangletoward object triangles keep a close direction to each other. Acollective control of secondary rays is enabled by this closeness and bydecoupling secondary rays from primary rays. The result is highcoherence of secondary rays.

FIELD OF THE DISCLOSURE

The present invention relates to generation of photorealistic real-timereflections from non-planar surfaces in hybrid ray tracing for anadvanced visual quality in video gaming, VR, AR, etc.

BACKGROUND

Ray tracing is a computer graphics technology capable of producing avery high degree of visual realism, higher than that of typical rastermethods, but at a greater computational cost. Ray tracing is superior toraster graphics by its capability to simulate a wide variety of opticaleffects, such as glossiness, specularity, radiosity, reflection andrefraction, scattering, soft shadows and more. True photorealism occurswhen the rendering equation is closely approximated or fullyimplemented. Implementing the rendering equation gives truephotorealism, as the equation describes every physical effect of lightflow. However, this depends on the available computing resources. Pathtracing, referred to as a Monte Carlo ray tracing is the physicallycorrect ray tracing. It gives an accurate simulation of real-worldlighting. Traditional ray tracers [Kajiya, J. T. 1986. The renderingequation. In Proc. SIGGRAPH] shoot rays through each pixel,stochastically scattering according to the profile of the intersectedobject's reflectance and continuing recursively until striking a lightsource. Repeated sampling for any given pixel in the image space willeventually cause the average of the samples to converge to the correctsolution of a rendering equation, making it one of the most physicallyaccurate 3D graphic rendering methods in existence. The prior art raytracing is one of the most computationally complex applications. Assuch, it is best suited for applications where the image can be renderedslowly ahead of time, such as in still images and film and televisionvisual effects, and is poorly suited for real-time animated applicationof augmented reality where the real time animation is critical.

Hybrid ray tracing (ray tracing interlaced with raster rendering) is adeferred rendering process based on raster rendering to calculate theprimary ray collision, while the secondary rays use a ray tracingapproach to obtain shadow, reflection and refraction effects. Thisapproach vastly improves ray tracing performance, not only because manyunnecessary traditional ray tracing tasks are avoided, but also becausea complete image is available in a demanded time, even if there is notenough time to finish calculations of all the visual effects. Thisfeature is valuable in video gaming, VR and AR where real time iscrucial, therefore the quality may be traded off for performance.

The concept of a hybrid Real-Time Raster and Ray Tracer renderer is notnew. Beck et al [Beck et al [Beck, S., c. Bernstein, A., Danch, D.,Frohlich, B.: CPU-GPU hybrid real time ray tracing framework (2005)]proposes a CPU-GPU Real-Time Ray-Tracing Framework. Beck proposal spreadthe traditional stages of ray tracing in independent tasks for the GPUand CPU. These render tasks can be summarized into three GPU renderpasses: a shadow map generation pass, a geometry identification pass anda blur pass.

Bikker [Bikker, J.: Real-time ray tracing through the eyes of a gamedeveloper. In: Proceedings of the 2007 IEEE Symposium on Interactive RayTracing, Washington, DC, USA, IEEE Computer Society (2007)] developed aReal-Time Path Tracer called Brigade, which divides the rendering taskseamlessly over both GPU and CPU available cores. Brigade aims theproduction of proof-of-concept games that use path tracing as theprimary rendering algorithm.

Pawel Bak [Bak, P.: Real time ray tracing. Master's thesis, IMM, DTU(2010)] implements a Real-Time Ray Tracer using DirectX 11 and HLSL.Similar to Beck's work, his approach also uses rasterization in order toachieve the best possible performance for primary hits.

Chen [Chen, C. C., Liu, D. S. M.: Use of hardware z-bufferedrasterization to accelerate ray tracing. In: Proceedings of the 2007 ACMsymposium on Applied computing. SAC '07, New York, N.Y., USA, ACM (2007)1046-1050] presented a hybrid GPU/CPU ray tracer renderer, where aZbuffered rasterization is performed to determine the visible trianglesat the same time that primary ray intersections are determined. The CPUreads the data back in order to trace secondary rays.

Sabino et al. [Thaler Sabino, Paulo Andrade, Esteban Gonzales Clua,Anselmo Montenegro, Paulo Pagliosa, A Hybrid GPU Rasterized and RayTraced Rendering Pipeline for Real Time Rendering of Per Pixel Effects,Univ. Federal Fluminense, Rio de Janeiro, Brazil, 2013] present aheuristic approach that select a subset of relevant objects to be raytraced, avoiding traversing rays for objects that might not have asignificant contribution to the real time experience.

An important strategy in real-time hybrid ray tracing, is the use of GPUfor raster techniques to improve the performance and a smart strategyfor prioritizing regions and objects that will receive the ray tracinglight effects. NVIDIA's OptiX [Parker, S. G., Bigler, J., Dietrich, A.,Friedrich, H., Hoberock, J., Luebke, D., McAllister, D., McGuire, M.,Morley, K., Robison, A., Stich, M.: Optix: A general purpose ray tracingengine. ACM Transactions on Graphics (August 2010)] is a general-purposeray tracing engine targeting both NVIDIA's GPUs and general-purposehardware in the current version. OptiX architecture offers a low-levelray tracing engine, a programmable ray tracing pipeline with a shaderlanguage based on CUDA C/C++, a domain-specific compiler and ascene-graph based representation. OptiX is a GPU only solution withremarkably good results for interactive ray tracing. Recently NvidiaOptiX has got a support by Nvidia RTX, which is a development platformfor hybrid ray tracing, for a special purpose hardware. It runs onNvidia Volta- and Turing-based GPUs, specifically utilizing anarchitecture for ray tracing acceleration.

Despite all the hybrid ray tracing developments, hybrid real-time raytracers on low-power devices do not exist in prior art. Theirapplicability on low-power devices, such as laptops, tablets, hand-heldmobiles, becomes more and more relevant. The likelihood of running raytracing on low power devices was forecasted only to the thirties: “ByMoore's law alone by 2032 we could be running real time ray tracing onmobile phones.” Jon Peddie, TechWatch, 27 Mar. 2018.

Real-time Ray Tracing (RTRT). Historically, ray tracing had beenreserved to off-line applications, such as computer-generatedphoto-realistic animated films. Real-time applications of video games,virtual and augmented reality had have to rely on rasterization fortheir rendering. RTRT is a hard-computational task, not only becauseeach pixel in the image must be calculated separately, but also becausethe final color of a single pixel can be affected by more than onerecursive ray. Another consideration is that ray tracing algorithmswaste from 75% to 95% of its execution time calculating intersectionpoints between rays and objects. RTRT has been enabled by Nvidia's RTXin 2018 (Alwani, Rishi. “Microsoft and Nvidia Tech to BringPhotorealistic Games with Ray Tracing”. Gadgets 360http://gadgets.ndtv.com/laptops/news/microsoft-dxr-nvidia-rtx-ray-tracing-volta-gpu-metro-exodus-1826988.Retrieved Mar. 21, 2018), facilitating a new development in computergraphics of generating interactive images that react to lighting,shadows, reflections by special purpose hardware. Nvidia's RTX is basedon traditional ray tracing algorithm accelerated by an on-chipsupercomputing hardware of closely 5000 cores. It comprises of a GPUhaving 4352 cores, AI denoiser utilizing 544 cores, and intersectiontests accelerator of 68 cores. The power requirement of a single RTX2080GPU is 250W, and the price starts at €418. Due to the high cost and highpower of RTX it is targeted at the high-end video games.

For video games, virtual reality and augmented reality, there is a greatneed to enable features that can be delivered only by the ray tracingtechnology, such as reflections, specifically reflections fromnon-planar surfaces that can't be achieved by the raster technology orby work-arounds. Reflection must be generated in real-time. Therefore,for the game industry there is a great need for a new hybrid real-timeray tracing technology, based on radical algorithmic improvements.

Reflections. In prior art's hybrid ray tracing the reflections aregenerated based on G-buffer (Luis Sabino et al., A Hybrid GPU Rasterizedand Ray Traced Rendering Pipeline for Real Time Rendering of Per PixelEffects, 2013). The G-Buffer is generated during the first stage byraster rendering, a “differed shading” stage. The basic idea behinddeferred shading is to perform all visibility tests before performingany lighting computations. Therefore, at first, visibility tests aredone by raster rendering, while shading is differed to a later stage,combined with ray tracing. The G-buffer produced by the deferred shadingstage contains information about optical properties of the underlyingmaterial of each pixel. Its contents are used to determine the need fortracing reflection/refraction rays. It is composed by reflectivity,index of refraction, specular exponent and opacity, respectively. Therays need to be traced from the surfaces only through the scene. Thisway enables to avoid trace of unnecessary rays in places where thematerial is neither refractive nor reflective. After differed shading isdone, the ray tracing algorithm starts with secondary rays and canfollow its own path. Any secondary ray generated will be traced againstscene in order to produce global illumination effects, such asreflections and refractions. The result of this stage can be understoodas the generation of a ray trace effects layer. This effects layer willbe blended to the image already generated, in order to improve itsvisual quality with global illumination effects.

According to the G-buffer method the secondary rays are a naturalextension of primary rays. Ray tracing that is carried-on by the chosensecondary rays suffer from the same difficulties of conventional raytracing: lack of coherence of secondary rays and images with stochasticnoise.

Generating fast reflections was described by Reuven Bakalash in U.S.Pat. No. 10,565,776: Method for Fast Generation of Path TracedReflections on a Semi-Reflecting Surface. His disclosure describes a newglobal illumination ray tracing, applied to augmented reality andvirtual reality. The Acceleration Structures of prior art are replacedby a new and novel device for carrying out the intersection betweensecondary rays and scene geometry in large groups of rays, gaining highspeed and lowering computational complexity. Its reduced powerconsumption is suitable to consumer level computing devices.

Accelerating structures. The most time-consuming tasks in ray tracingare intersection tests between millions of rays and millions ofpolygons. They are partly relieved by use of acceleration structures(AS) which are huge binary trees, specifically structured for the scenespace. Every single ray is traversed across an accelerating structure(e.g. K-trees or BVH trees), seeking polygons for intersection. Thesetraversals become a major time-consuming task—they typically take over70% of the image generation time.

The prior art AS based on binary trees (e.g. BVH) are basically static.Their reconstruction is typically more time consuming than rendering.The construction time depends on the scene size and polygon resolution.E.g. building an acceleration data structure of highly complex geometryis prohibitively expensive or imposed restrictions on how the geometryhad to be modelled. Hanika et al. addresses this difficulty in U.S. Pat.No. 8,570,322, Method, system, and computer program product forefficient ray tracing of micropolygon geometry. They teach a firsthierarchy of surface patches which is ray traced to identify which arepotentially intersected. Then the potentially intersected patches aredecomposed, on-demand, into a set of subobjects of micropolygons and asecond hierarchy is established in order to accelerate ray tracing.Shaders that operate on this second hierarchy of micropolygons canprocess an entire grid of them at once in SIMD fashion.

Small scene changes may need only an update of AS, however, a majormodification of the scene necessitates a reconstruction of theacceleration structures. Big scenes are posing a particular difficultydue to very long reconstruction times. Reuven Bakalash et al. addressesthis problem in U.S. Pat. No. 10,380,785 titled: Path Tracing MethodEmploying Distributed Acceleration Structures, by distributing theacceleration structure. The traversal task in a path tracing system isdistributed between one global acceleration structure, which is centralin the system, and multiple local acceleration structures, distributedamong cells, of high locality and of an autonomous processing.Subsequently, the centrality of this critical resource is reduced,lessening bottlenecks, while its parallelism is improved.

There are two major drawbacks associated with the use of staticacceleration structures; (i) traversals of these structures aretime-consuming, challenging the real-time requirements, and (ii) theymust be repeatedly reconstructed upon scene changes, which contradictswith real time skinned animation. Reconstructing static accelerationstructure is a computationally intensive task preventing real-timeanimation.

There is thus a need for addressing these and/or other issues associatedwith the acceleration structures of prior art.

Lack of ray coherence of secondary rays. Coherence of rays is the keyfor efficient parallelization of ray tracing. In prior art ray tracingthe primary and shadow rays are coherent. This coherence is exploitedfor efficient parallel processing: traversing, intersecting, and shadingby packets of coherent rays. They work well for nearby primary rays,since these rays often traverse similar parts of the accelerating datastructure. Using this approach, we can reduce the compute time by usingthe conventional SIMD mechanisms of GPUs and CPUs for multiple rays inparallel, reducing memory bandwidth by requesting data only once perpacket, and increasing cache utilization at the same time. This worksfine for primary rays that originate from the camera. Unfortunately, itis not possible to use ray packets effectively with rays of an advancedorder (secondary, ternary, etc.). The primary reason is the advancedorder rays bounce in different direction losing coherence. Moreover,there is an intentional randomization of rays for diffuse reflections.Reorganizing secondary rays to form bundles with higher coherenceratios, are practiced by the prior art. But this kind of regrouping is aquite expensive operation since it involves a scatter/gather step, whichmay result in only a slight frame rate improvement when reordering isapplied.

Sadegi et al. [Iman Sadeghi, Bin Chen, and Henrik Wann, Coherent PathTracing, Jensen University of California, San Diego, 2009], developed atechnique for improving the coherency of secondary rays. This techniqueuses the same sequence of random numbers for generating secondary raysfor all the pixels in each sample. This improves the efficiency of thepacket tracing algorithm but creates structured noise patterns in theimage.

Improving coherency in ray tracing is addressed by Reuven Bakalash etal. in U.S. Pat. No. 10,410,401 titled: Spawning Secondary Rays in RayTracing from Non-Primary Rays. He describes a novel way of generatingcoherent secondary rays for a global illumination ray tracing. TheAcceleration Structures of prior art are replaced by a new and noveldevice of carrying out the intersection between secondary rays and scenegeometry in large groups of rays, gaining high speed and loweringcomputational complexity.

There is thus a need for addressing these and/or other issues associatedwith lack of coherency in ray tracing of prior art.

Noisy images. A path tracer continuously samples pixels of the screenspace. The image starts to become recognizable after only a multiplesamples per pixel. Rays are distributed randomly within each pixel inscreen space and at each intersection with an object in the scene a newreflection ray, pointing in a random direction, is generated. After somenumber of bounces, each ray eventually exits the scene or is absorbed.When a ray has finished bouncing about in the scene a sample value iscalculated based on the objects the ray bounced against. The samplevalue is added to the average for the source pixel.

The random components in ray tracing cause the rendered image to appearnoisy. The noise decreases over time as more and more samples arecalculated. The defining factor for render quality is the number ofsamples per pixel (SPP). The higher SPP you have in a rendered image theless noise will be noticeable. However, the added quality per sampledecreases the more samples you have already (since each sample is justcontributing to an average over all samples).

Only converge of many subsequent frames reduces the final image noise.The image to converge and reduce noise to acceptable levels usuallytakes around 5000 samples for most path traced images, and many more forpathological cases. Noise is particularly a problem for animations,giving them a normally unwanted “film-grain” quality of randomspeckling.

An object of at least one of the embodiments of the disclosure is toenable a use of real-time photorealistic reflections in video games, VRand AR, and to solve a lack of relevant technology in the related art.

SUMMARY

The embodiments of the disclosure generate photorealistic reflections innon-planar reflective surfaces in hybrid ray tracing. In order toaccommodate real-time skinned animation a genuine dynamic accelerationstructure (DAS) is employed. It is characterized by high locality,wherein scene changes are updated locally in one of its hierarchies,without effecting other locations in the structure. Each scene object inDAS can move independently of other objects, allowing autonomous updatesat its own hierarchy, without effecting its neighboring hierarchies.Therefore, fast reconstructions of entire accelerating structures ofprior art is replaced by low-cost updates.

The efficiency of DAS traversals is improved by a double step traversal;first with a geometric volume, e.g. triangular pyramid to eliminatenegative intersection tests, and then with secondary rays for efficientand accurate intersection tests of the leaf hierarchy.

Coherence of secondary rays, a long-felt need in ray tracing, isachieved by handling secondary rays with collective origin andcollective destination. Such a coherence attains high parallelism andhigh utilization of GPU.

BRIEF DESCRIPTION OF THE DRAWINGS

The disclosure is herein described, by way of non-limiting examplesonly, with reference to the accompanying figures and drawings, whereinlike designations denote like elements. Understanding that thesedrawings only provide information concerning typical embodiments of thedisclosure and are not therefore to be considered limiting in scope:

FIG. 1. An exemplary setup of a non-planar reflective surface andgeometric objects.

FIG. 2. A triangle of a non-planar reflective surface related togeometric objects.

FIG. 3. Object's triangle projected onto triangle of a non-planarreflective surface.

FIG. 4. Object's triangle related to triangle of a non-planar reflectivesurface. No projection.

FIG. 5. Dynamic accelerating structure of four geometric objects.

FIG. 6. Pre-animated scene.

FIG. 7. Skin-animated scene.

FIG. 8. DAS of the pre-animated scene.

FIG. 9. DAS of the skin-animated scene.

FIG. 10. Frustum from receiver triangle.

FIG. 11. Step of visualization traversal.

FIG. 12. Step of a secondary ray traversal.

FIG. 13. Flow chart of analytical embodiment.

FIG. 14. Flow chart of another embodiment.

FIG. 15. Prior art. Computing system in accordance with the embodiments.

DETAILED DESCRIPTION OF THE DISCLOSURE

An object of at least one of the embodiments of the disclosure is toenable a use of photorealistic reflections in video games by solving thelack of relevant technology in the related art. Real time reflections,from planar and non-planar, reflective and glossy surfaces are enabledthrough a real-time hybrid ray tracing technology.

Mirror reflections from non-planar surfaces. A non-planar surface,unlike a planar one, is lacking a uniform direction. It is constructedof triangles which are planar elements. Each triangle reflects the sceneaccording to its normal direction. The method of disclosure is meant togenerate a physically correct reflection of objects in a non-planarsurface. An exemplary setup is shown in FIG. 1. Two target objects 11,12 would reflect in a non-planar surface 10 consisting of multiplereceiver triangles.

According to one embodiment each receiver's triangle is handledseparately to generate its own physically correct reflection. A finalgathering of the partial reflections results in an aggregate non-planarsurface reflection.

FIG. 2 shows 20 one of the surface's triangles. The two target objects26, 27 consist of meshes of emitting triangles. In order to calculate acorrect reflection in the receiving triangles, a normal of each suchtriangle must be calculated. Then secondary rays are shot from accordingto the normal direction from each receiver triangle toward emittingtriangles of the objects.

A receiver triangle's normal is evaluated based on the normals of itsthree vertices. Each such vertex is shared with neighboring triangles.Therefore, due to non-planarity, each vertex has its own normaldirection. A triangle's normal direction N is calculated by averagingthe normal directions of its three vertices.

Once the triangle's normal is known, a mirror surface 21 through themid-point of the triangle is obtained, and then the mirror eye 25, whichis the camera 28 reflection in the mirror surface, is defined. Themirror eye serves as an origin of a frustum projection. The frustrum isshaped by the receiver's triangle 20 and directed toward the targetobjects 26 and 27. The frustum defines the scene visibility from thetriangle. In the example of FIG. 2 the object 26 is entirely visible,while object 27 is only partly visible. Thus, portion of the trianglemesh of 27 remains outside the frustum.

An analytical embodiment of generating reflection is illustrated in FIG.3. A receiving triangle 30, an emitting triangle 33, and the finalemitter's reflection 34 in the receiver, are shown. The task ofreflecting an emitting triangle onto receiving triangle applies only tothe visible triangles of the object mesh. The frustum visibility testidentifies the emitting candidates for reflection. The detailedvisibility test is described hereinafter.

Once the emitter triangle 33 passed the visibility test, it is handledfor reflection. It is sampled for color, light, material, texture, etc.,and projected onto receiver's surface by its three vertices 35, 36, 37.Emitter triangles that fall partly outside the receiver, are clipped toreceiver's boundaries. Then the projection is shaded according to thesampled light values. Then these secondary light values are merged withtheir respective primary hit points on the receiver triangle, toreconstruct the final reflected light. Finally, the results aredelivered, together with other receiver triangles, to the image pixelson the screen.

Another embodiment utilizing secondary rays, FIG. 4, reflects theemitting triangle in the receiving one by full use of secondary rays. Areceiving triangle 30, and an emitting triangle 33 are shown. Thereflection task applies only to those triangles of the target objectmesh that passed the visibility test and have been identified ascandidates for reflection.

As shown back in FIG. 2, secondary rays cover the frustum area from thereceiver 20 and up along the dashed lines 29. Secondary rays are meantto make an accurate examination of the visible mesh, triangle bytriangle. Each secondary ray performs intersection tests with emittertriangles, and the intersected triangles are sampled for color, light,material, texture, etc. The sampled data is merged with the respectiveprimary hit points to merge the sampled light values of the primary andsecondary rays, for full reconstruction of the reflection. Finally, theresults are delivered, together with other receiver triangles, to theimage pixels on the screen.

Dynamic acceleration structure. Acceleration structures (AS) in raytracing require fast construction and fast traversals. Unfortunately,these two requirements contradict each other. BVH acceleratingstructures are the most widespread in prior ray tracing. The morehierarchical levels the faster is the traversal, but the constructionbecomes more time expensive. Fast construction and fast traversals areimperative for real-time ray tracing, and specifically for real-timeskinned animation.

In order to accommodate real-time skinned animation, the disclosureemploys a genuine dynamic acceleration structure (DAS) with highlocality, in which the changes in the scene, such as skinned animationof an object, are updated locally, in a specific location within the DAShierarchy, without effecting other locations in. The DAS is constructedonly once, and then only the required per-frame updates are done. Thus,the need for fast construction is replaced by the low-cost updates.

The DAS is established of a separate hierarchy per each scene-objectsuch that each subsequent hierarchy level comprises sub-meshes of thepreceding level, wherein all the sub-meshes are bounded by boundingboxes. At the bottom of each hierarchy is a leaf level of triangles.

For fast traversals, an embodiment is based on an early visibility testthat eliminates redundant intersection tests, and only thensecondary-ray intersection tests. In the first phase a visibility testof target objects is done by traversing the mesh and sub-meshhierarchies with a geometric volume, e.g. pyramidoid. Non-visibleobjects or part thereof are dropped off, saving redundant intersectiontests. During the second phase only the leaf hierarchy of triangles istraversed with secondary rays. In bottom line, the amount ofintersection tests is reduced.

An example of a DAS structure is shown in FIG. 5. The DAS ishierarchically structured, wherein in the first, the highest hierarchy50, it contains bounding boxes of geometric objects. The examplecomprises meshes of four target objects 52-55. The next hierarchy isformed of sub-meshes 51. The same way an arbitrary number of additionalhierarchies can be formed, wherein each successive hierarchy comprisessub-meshes of the previous hierarchy. The leaf hierarchy 59 is formed oftriangles. The DAS may be built in a preprocessing operation priorstarting rendering frames, or in run time.

The DAS is structured differently from bounding volume hierarchy (BVH)of prior art. Its top hierarchy starts directly with the geometricobjects, no single bounding volume at the top of the tree. Eachgeometric object is independent of other objects, allowing autonomousupdates at its own hierarchy, without effecting its neighboring objects.

In the animation art a skinned animation is typically represented in twoparts: a surface representation used to draw the character (called skinor mesh) and a hierarchical set of interconnected bones (called theskeleton or rig) used to animate the mesh. The intention is to controlthe first part only, graphics deformation of the mesh data. When suchdeformation of an object occurs, its surface representation isaccordingly updated within its main mesh, sub-meshes and their boundingboxes, keeping locality and autonomy, without effecting the neighborobjects or the hierarchy of the overall structure. Such autonomy of eachgeometric object makes the DAS dynamic allowing efficient and fastupdate of dynamic scenes.

The principle of locality of the skinned animated updates in the dynamicacceleration structure is demonstrated in FIGS. 6-9. In FIG. 6 threeobjects are shown 60, 61, and 62. The mid object 62, is a human figurineat its original position. Then it is shown skinned animated in FIG. 7,when transferred and transformed. The figurine 72 moved to another placeand raised its hands. FIG. 8, referring to FIG. 6, shows the originalpre-animated DAS. The figurine is represented by the mid column. Its tophierarchy comprises bounding box 86 of all its parts 80-85. The secondhierarchy contains bounded sub-meshes 80′-85′ of the parts, while thebottom hierarchy comprises the triangles. The two other columnsrepresent the hierarchies of two other objects, which remain static.

Frustum intersection test. Intersection tests between millions of raysand millions of polygons is the most time-consuming task in ray tracing.In prior art every single ray must traverse across hierarchies of anaccelerating structure seeking for intersections. The efficiency isreduced by making intersection tests that end up with negative results.According to an embodiment the efficiency is improved by eliminatingmost of the negative tests. A preliminary visibility test is introducedfor culling invisible meshes. The visibility test is done by traversingthe mesh hierarchies of the accelerating structure with a triangularpyramid that bounds the frustum. Parts that fall out of the frustumpyramid are sorted-out because of the non-relevant intersection tests.

FIG. 9, referring to FIG. 7, demonstrates the post animated DAS. Thebounding box 96 was updated following the displacement of sub-parts 91,92. In the second hierarchy the sub-meshes 91′ and 92′ and theirbounding boxes undergo an update. All the updates went local, withouteffecting other sub-meshes of the same object or other objectshierarchies. This locality of updates makes the DAS a dynamic datastructure, enabling real-time skinned animation. Such real-timeanimation is critical for delivering high visual quality to video games,virtual reality and augmented reality.

According to an embodiment the efficiency of intersection tests isimproved by introducing a preliminary visibility test to eliminateinvisible parts of objects, which have no chance of positiveintersection tests. As shown, the frustum of FIG. 10 has a shape of atriangular pyramid consisting of three sides 104, 105 and 106, cut bytheir near end 103 (the receiver triangle) and their far end 100. Thefrustum vertices are guided by the receiver triangle's normals 107, 108,109. As mentioned before, the receiver triangle's normals may bedifferent from each other, where each vertex shares its normal withneighbouring receiver triangles of the non-planar surface. The frustumhas a shape of a triangular pyramid based on the receiver triangle. Areceiving triangle 103 and three emitting objects 100, 101 and 102 areshown in this exemplary setup. The object 101 is fully visible,therefore eligible to all intersection tests. The second object 102falls partly outside the frustum, its sub-meshes that fall entirelyoutside would be eliminated. The third object 110 that falls outside thefrustum, would not be tested for intersections.

FIG. 11 illustrates the process of sorting out objects and theirsub-meshes in DAS (111) that are laying entirely outside the frustum.Polygons 104, 105, 106 represent the three frustum's sides. Each ofthese walls is used in a row to traverse the mesh and sub-mesh hierarchylevels of the DAS bounding boxes, to cut out the out-of-frustrum objectsand parts thereof. This traversal occurs only with the mesh levels ofthe DAS, from the upper level down to the last sub-mesh level, rightabove the triangle level. An output of the first wall's traversal 104 isused as an input to the next wall's traversal 105, and then to wall'straversal 106. Each traversal sorts out bounding boxes of object meshesor of sub-meshes that fall entirely on the opposite side of the frustum.During the frustum-test the triangle level 59 (FIG. 5) stays out oftraversal. The result of the frustum-test is a list of relevantsub-meshes 110 of the last mesh hierarchy above the triangle level, ascandidates for intersection tests.

According to one embodiment, for accuracy and efficiency, only the lasthierarchy level is traversed with secondary rays. That level oftriangles is traversed separately. Once the frustum traversal step hasbeen finalized, a subsequent traversal step takes place. It is based ontracing secondary rays from the receiver triangle toward the emittingobjects. In this traversal step, only the sub-meshes that survived thevisibility test remain active. Their triangles are tested by secondaryrays for intersection.

This second step is shown in FIG. 12. The DAS structure 121 is sparse,comprising only the candidate meshes, sub-meshes and triangles thatsurvived the visualization test. Only the lowest sub-mesh hierarchy,directly above the triangle level, is shown 122. The candidate triangles123, shown in the bottom level of DAS, are the candidates forintersection tests.

According to the analytical embodiment a single intersection test foreach emitting triangle suffices for choosing a triangle. No additionalintersection tests for the triangle are needed. The emitter triangle istaken for an analytical projection on the receiving triangle and forshading. Each secondary ray of a receiving triangle would traversethrough the candidate emitting triangles. The first time an emittingtriangle is found hit by a ray, the triangle is removed from the list ofcandidate triangles and is projected on the receiving triangle. The listis shrinking throughout the intersection tests speeding up theintersection process.

The embodiment of analytical intersection tests is shown in the flowchart of FIG. 13. In stage one the geometric objects or part thereof,are clipped out of by the frustum test, eliminating intersection teststhat would occur outside the frustum. Stage two, the remaining emittingtriangles undergo an intersection test by secondary rays, and ifintersected once, then an analytical projection of that emittingtriangle on the receiving triangle is done, saving subsequentintersection tests.

First 131 the highest hierarchy bounding boxes of the DAS structure (111of FIG. 11) are intersected with each of the three clipped frustum'ssides 104-106. At each intersection the geometric objects that areentirely outside the frustum are dropped from candidacy. For objectsremaining at least partly inside the frustum their sub-meshes of nexthierarchy are taken for intersection with the frustum sides. Again, foreach intersection these sub-meshes that are entirely outside the frustumare dropped 132. The frustum intersection test is descending along thesub-mesh hierarchy 133, wherein the last intersection is done directlyabove the triangle level. Only the triangles that belong to thesub-meshes that passed at least partly the visibility test are kept inthe list of active sub-meshes 110. The second stage starts at the block134. Secondary rays are shot from the receiver's triangle at activetriangles. Each secondary ray makes intersection tests with allcandidate triangles. Per each triangle that is hit by a secondary ray,remove the triangle from the candidate list and project it analyticallyonto receiver triangle. Then 135, once a secondary ray hits a triangle,the triangle is sampled for material, color, light, etc., and ananalytical projection of the triangle onto the receiver's triangle isdone (see FIG. 3). No additional intersection tests between secondaryrays and the intersected triangle are done. The redundant intersectiontests are saved, improving efficiency.

According to another embodiment, all secondary rays of a receivingtriangle would travers through all the candidate triangles 110, seekingfor intersection. The length of the candidate list remains stable. Theintersected triangles are sampled for light values at the point ofintersection. This embodiment is flowcharted in FIG. 14. The first stageof eliminating objects or part thereof that reside out of the frustum isidentical to the analytical embodiment 141-143. In stage two, theremaining emitting triangles undergo an intersection test with allrelevant secondary rays.

First 141 the highest hierarchy bounding boxes of the DAS structure (111of FIG. 11) are intersected with each of the three clipped frustum'ssides 104-106. At each intersection, geometric objects that are notvisible, entirely outside the frustum are dropped. For objects remainingat least partly inside the frustum their sub-meshes of next hierarchyare taken for intersection with the frustum sides. Again, for eachintersection the sub-meshes that are entirely outside the frustum aredropped 102. The frustum intersection test is done down the sub-meshhierarchy 143, wherein the last intersection is done at the hierarchylevel directly above the triangle level. Only the triangles that belongto the sub-meshes that passed at least partly the inside-frustum testare kept in the candidacy list 110. The second stage starts at block144. Secondary rays are shot at active triangles 123 from receiver'striangle. Each secondary ray makes intersection tests with all activetriangles. Once secondary ray hits a triangle, it brings a sample ofmaterial, color, light, etc. from its closest hit (multipleintersections may occur for a single ray) to the receiver's triangle145.

In both embodiments, the light values sampled at the emitting trianglesare merged with their respective primary hit points to reconstruct thereflection of the object in the receiving triangle. Finally, the mergedresults are delivered to the image pixels on the screen, together withother receiver triangles.

Coherence of secondary rays. A coherence of secondary rays is achievedin the disclosure due to the collective way they are created andprocessed. In prior art the secondary rays are created in a directcontinuation of primary rays, they bounce where the primary rays hit.According to one embodiment, secondary rays are decoupled from primaryrays. They are created and processed independently (e.g. for reflection,color bleeding, ambient occlusion, shadow, etc.) of primary rays.However, when done, the data generated by a secondary ray is merged withthe data of its counterpart primary ray and transferred to thecorresponding image pixel. That is, primary and secondary rays areworking asynchronously to each other, but their collected data isfinally merged. The same applies to the relation between ternary andsecondary rays, and for higher order of rays as well.

The decoupling between primary and secondary rays allows handling ofsecondary rays in a package, such as a frustum projection of rays104-106 in FIG. 10. They all are shot from a joint origin, like thereceiver triangle 103, often hitting similar scene objects e.g. 101 and102, and traversing similar parts of the accelerating data structure,e.g. 123 of FIG. 12. The result of joined pathways of secondary rays ishigh coherence, which is a long-felt need in ray tracing. The secondaryrays of the disclosure are coherent because they all belong to the samefrustum, that is they all commence at the same triangle and take aboutthe same direction. The coherent handling of secondary rays attains highparallelism and high utilization of GPU.

Computing system. FIG. 15 shows a computing system 156 for an efficientray tracing in accordance with the embodiments of the disclosure. Avarious architecture and functionality of the previous embodiments canbe implemented. As shown, a system is provided including at least onehost processor 150 which connects to communication bus 155. The systemalso includes host memory 151. Software and data are stored in the mainmemory which may take the form of RAM. The system also includes agraphics system, having a graphics processor (GPU) 152 and a display153. The GPU has a plurality of shaders, rasterization module, etc. Theembodiments are best suited for GPU. The coherence of secondary raysachieved in the disclosure attains high utilization of GPU.

In one embodiment the graphics processor is a discrete GPU having itsown video memory and hosted by a CPU. In another embodiment the GPU maybe integrated in a single chip with CPU cores, sharing the memory.Similarly, in one embodiment, the foregoing modules may be situated in asemiconductor platform like an FPGA and/or other reconfigurable device.As an option these devices may be in-socket devices. The system 156 mayinclude a secondary storage 154 as well. Further, while not shown, thesystem 156 may be coupled to a network (e.g. internet, LAN, WAN) forcommunication purposes.

The methods provided can be implemented in a general purpose computer, aprocessor, or a processor core. Suitable processors include, by way ofexample, a general purpose processor, a special purpose processor, aconventional processor, a digital signal processor (DSP), a plurality ofmicroprocessors, one or more microprocessors in association with a DSPcore, a controller, a microcontroller, Application Specific IntegratedCircuits (ASICs), Field Programmable Gate Arrays (FPGAs) circuits, anyother type of integrated circuit (IC), and/or a state machine. Suchprocessors can be manufactured by configuring a manufacturing processusing the results of processed hardware description language (HDL)instructions and other intermediary data including netlists (suchinstructions capable of being stored on a computer readable media). Theresults of such processing can be maskworks that are then used in asemiconductor manufacturing process to manufacture a processor whichimplements features of the disclosure.

While various embodiments have been described above, it should beunderstood that they have been presented by way of example only, and notlimitations. Thus, the breath and scope of a preferred embodiment shouldbe defined only in accordance with the following claims and theirequivalents.

What is claimed is:
 1. A computer-based method for hybrid ray tracing ofnon-planar surface reflections, created by coherent secondary rays shotat a scene's geometric objects, wherein secondary rays are decoupledfrom primary rays, comprising: a) identifying, independently of primaryrays, a non-planar reflective surface in a scene to reflect plurality ofgeometric objects; b) identifying the plurality of geometric objects inthe scene; and c) at each triangle of the non-planar reflective surface:creating a frustum directed at the plurality of geometric objects,wherein vertices of the frustum are guided by normal vectors of thetriangle; generating a packet of secondary rays bounded by the frustumand shot from the triangle toward the plurality of geometric objects;and traversing the packet of secondary rays in an accelerating structureas a packet of the coherent secondary rays, while upon hitting atriangle of a geometric object by a secondary ray, the triangle issampled and projected onto the triangle of the non-planar reflectivesurface.
 2. The method of claim 1, wherein the secondary rays comprise acoherent packet of rays.
 3. The method of claim 1, wherein the frustumhas a shape of a triangular pyramid based at the triangle of thereflective surface.
 4. The method of claim 1, wherein the frustum isshot from the triangle toward the geometric objects for exploringvisibility.
 5. The method of claim 1, wherein the traversing improvesefficiency of intersection tests by culling invisible geometric objects.6. The method of claim 1, wherein all secondary rays shot from eachtriangle of the non-planar reflective surface traversing theaccelerating structure have one common origin and mutually closedestinations.
 7. The method of claim 6, wherein the common origin andthe mutually close destinations make the secondary rays coherent.
 8. Themethod of claim 7, wherein the coherence of the secondary rays is gaineddue to the collective way they are created, allowing their processing inpackets.
 9. The method for hybrid ray tracing of claim 1, wherein theshooting of secondary rays is done independently of an image generatedby the primary rays.
 10. The method of claim 1, wherein the secondaryrays are created and processed independently of a rasterized image,while their resulting data is merged with the rasterized image.
 11. Themethod of claim 10, wherein the way of processing the secondary raysindependently of the rasterized image applies also to rays of higherorder.
 12. The method of claim 1, wherein a reconstruction of theaccelerating structure is performed by local updates.
 13. The method ofclaim 12, wherein the local updates of the acceleration structure enablefast animation of each of the geometric objects or part thereof in thescene.
 14. The method of claim 1, wherein the coherence of the secondaryrays enables high parallelism of traversals through hierarchies of theaccelerating structure.
 15. The method of claim 14, wherein thecoherence of the secondary rays enables high utilization of a graphicsprocessing unit.